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DEVELOPMENT  OF  INTEREST  SCALES  TO  PREDICT  PERFORMANCE 
IN  CONSTRUCTION  AND  MECHANICAL  MOS 

He  search  to  improve  enlisted  classification  lias  Included  an  effort 
to  measure  the  oacujiational  interests  and  goals  of  the  new  recruit  in 
order  to  enhance  prediction  of  MOS  training  and  Job  performance.  As  a 
part  of  this  effort  in  the  broad  mechanical  domain,  the  Army  Job  Activ¬ 
ities  Questionnaire -Mechanical  (AJAQ-M)  was  constructed  and  administered 
to  samples  of  general  Army  enlisted  input.  Initial  findings  on  the 
relationships  between  specific  Job  activity  valences,  preferences  us  to 
Job  conditions,  expressed  interest  in  givpn  Jobs,  and  related  pre*^ervlce 
experience  have  been  reported  elsewhere*  (He law  uhA  lCnf!  ;  19*>ft-)  The 
earlier  report  discussed  motivational  factors  and  scales  obtained  from 
internal  item  analysis  on  AJAQ-M..  The  next  step  was  to  administer  the 
measure  to  samples  of  input  Jt®  selected  training  courses  so  as  to  obtain 
item  validity  coefficients'  for  both  training  and  Job  performance  evalua¬ 
tions.  present  publication  presents  the  findings  from  the  item 

analysis  and  describes  the  development  of  empirically  valid  scales  which 
are  to  be  validated  on  independent  samples  in  the  next  research  stage. r 

\ 


METHOD  OF  ATTACK 

The  AJAQ-M  was  administered  to  samples  of  men  entering  advanced 
individual  training  in  five  MOS  (Table  1.).  Variables  consisted  of  the 
179  items  of  the  instrument,  which  were  of  four  types: 


Number  of 

Types  of  Content  Section  Items  Alternatives 


Specific  Job  activity  valences 

I 

100 

4 

Preference  for  job  conditions,  require¬ 
ments 

II 

50 

2 

Valences  of  described  Jobs 

III 

11 

r\ 

Experience  in  particular  Job  activities 

IV 

18 

o 

c. 

Each  item  was  correlated  with  the  two  criteria:  MOS  training  course 
grade  and  Job  performance  ratings  about  7  or  8  months  after  completion 
of  training. 


i->  Heime,  W.  H.  and  Katz,  Aaron.  Construction  of  Experimental  Interest 
Measures  for  Enlisted  Classification.  Research  Memorandum  65-2. 

U.  S.  Army  Personnel  Research  Office.  May  1965* 


Table  1 


SAMPLES  USED  TO  DERIVE  SCALES  OF  VALID  ITEJ6  FROM  ARMY  JOB 
ACTIVITIES  QUESTIONNAIRE,  MECHANICAL  (AJAQ-M) 


MOS 

Number  of  Cases 

Designation 

No. 

Training  Criterion  Job  Criteria 

Pioneer 

12A 

159 

116 

Wireman 

56A 

104 

75 

Lineman 

56C 

186 

150 

Welder 

44C 

117 

94 

Engineer  Missile 

Equipment  Specialist 

62C 

105 

81 

In  addition  to  identifying  the  more  valid  items  in  each  criterion 
sample,  the  relationship  of  item  validity  to  item  factorial  content  as 
derived  in  the  earlier  analysis  (Helme  and  Katz,  op.  cit.)  was  investi¬ 
gated.  Since  the  factor  scales  represented  psychological  constructs, 
validity  estimates  of  sets  of  items  from  homogeneous  scales  might  be 
expected  to  be  more  stable  than  validity  estimates  of  sets  of  items 
selected  purely  on  the  empirical  basis  of  single-item  validity  coeffi¬ 
cients. 


RESULTS 


Item  validity  coefficients  were  obtained  for  each  item  against  ten 
criterion  measures — training  and  job  evaluations  in  each  of  the  five  MQS. 
Initial  analysis  showed  that  there  were  few  items  with  consistent  validity 
across  all  samples,  or  even  with  validity  consistent  for  both  the  training 
and  Job  criteria  on  a  single  MOS.  This  result  raised  a  question  whether 
the  item  coefficients  on  single  samples  were  sufficiently  stable  to  afford 
a  basis  for  a  scale,  particularly  in  view  of  the  facts  that  (l)  the  Job 
criterion  samples  were  composed  entirely  of  members  of  the  training 
criterion  samples,  and  (2)  the  training  and  Job  criteria  in  a  given  MOS 
were  presumably  positively  correlated.  Moreover,  on  a  practical  basis, 
little  was  to  be  gained  by  deriving  10  different  "valid"  scales. 

In  an  effort  to  obtain  empirical  scales  with  reasonable  prospect  of 
stable  validity  from  sample  to  sample,  the  relation  of  item  validity  to 
factorial  content  was  next  explored,  using  data  on  the  100  items  of 
Section  I  (specific  Job  activity  valences).  When  attention  was  restricted 
to  the  items  of  a  single  factor  set,  there  appeared  to  be  consistent 
results.  These  factor  sets  are  shown  in  Table  2.  Within  the  Power  set 


of  items,  consistent  validity  for  Job  performance  in  MOS  131  (Pioneer), 

ViA  (Wireman),  and  44C  (Welder)  was  found  for  the  factor  subsets  of  Manual 
Construction,  Construction  Equipment  Operation,  and  Carpentry  Tools,  while 
the  subset  of  Installation  and  Repair,  and  the  items  loading  on  the  Power 
factor  only,  were  generally  of  lower  validity  (Table  5).  On  the  other 
hand,  for  the  criterion  of  training  performance  in  MOS  with  some  electrical 
content,  }oA  (Wireman),  3LC  (Lineman),  and  to2C  (Engineer  Missile  Equip¬ 
ment  Specialist),  the  Danger  and  Technical  items  tended  toward  better 
validity  while  the  Discomfort  factor  items  were  markedly  less  valid. 
Miscellaneous  items  not  loaded  on  any  of  these  factors  showed  consistently 
low  validity  for  both  criteria.  Accordingly,  two  empirically  valid  scales 
were  derived:  A  Mechanical  Job  Performance  Scale  of  20  items  selected 
from  the  Power  factor  sets,  and  an  Electrical  Training  Performance  Scale 
of  20  items  selected  from  the  Danger  and  Technical  factor  sets,  -w 
scales  are  shown  In  Tables  A-l  and  A-2. 

The  items  of  Section  II,  Job  condition  preferences,  were  analyzed 
next.  Some  consistency  of  validity  was  found  for  the  electrical  train¬ 
ing  criteria,  but  the  Job  performance  criteria  did  not  yield  consistent 
results.  Instead,  a  moderate  agreement  in  validity  coefficients  appeared 
between  the  two  MOS  involving  heavy  construction:  Pioneer  (l2A)  and 
Lineman  (36C). 

Another  Army  Job  Activities  Questionnaire  had  been  constructed  and 
tried  out  for  the  medical  and  chemical  fields — Biochem  (AJAQ-BC).  This 
instrument  contained  the  same  Section  II  as  the  AJAQ-M,  inasmuch  as 
general  Job  conditions  may  be  found  across  a  wide  range  of  Jobs.  Item 
validity  data  for  training  performance  in  four  MOS  samples  were  examined: 

(Chemical  Operations  Apprentice),  91^  (Medical  Corpsman),  91B 
(Medical  Specialist),  and  higher  skill  MOS  in  the  91  (Medical  Care  and 
Treatment)  and  92  (Laboratory  Procedures)  career  groups.  High  consistency 
of  validity  was  found  for  the  lower  skill  MOS  (Chemical  Operations  Appren¬ 
tice  and  Medical  Corpsman)  and  moderate  consistency  between  these  and 
one  or  both  of  the  MOS  groups  in  the  A JAQ-M  analysis.  Moreover,  the 
valid  responses  tended  to  be  in  the  same  direction  for  both  the  chemical - 
medical  and  the  electrical  and  construction  samples;  preference  for 
the  higher  technical  level  of  achievement  was  positively  related  to  the 
performance  criteria.  Therefore,  a  Medical -Technical  Scale  of  20  items 
was  assembled  (Table  A -3).  The  consistency  of  level  of  validity  across 
the  three  criteria  is  shown  in  T:  ble  4.  Of  the  items  selected  across 
these  criteria,  are  above  the  median  validity  for  chemical -medical 
MOS  and  about  two-thirds  are  above  the  median  for  electrical  and  construc¬ 
tion  MOS.  Half  the  items  selected  are  above  the  70th  percentile  of  the 
respective  distribution  of  validity  coefficients.  Incidentally,  there 
was  little  relation  to  the  factor  scale  obtained  from  earlier  analysis 
of  Section  II  (Mechnical  Blue  Collar  vs.  Technical  White  Collar ) --there 


One  highly  valid  item  fro*  the  Discomfort  set  was  included  in  place  of 
possible  borderline  items  from  the  other  two  sets. 
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Table  2 


FACTOR  SCALES  DERIVED  FROM  INTERNAL  ANALYSIS  OF  AJA^-M 


Major 

Section  Factor 

Number  of 
Items 

Average 

Loading 

Secondary 

Factor 

Average 

Loading 

11 

Power 

8 

.56 

Manual  Const 

.32 

Power 

10 

.54 

Install  & 
Repair 

•  32 

Power 

6 

.56 

Const  Equip  Op  .35 

Power 

4 

.62 

Carpentry 

Tools 

.35 

Danger  10 

Tec  Iml  cal  14 

Discomfort  9 

II  Mechanical  vs.  Technical  17 

III  Manual  Skills  3 

Heavy  Construction  3 

Installation  t  Maint  4 

IV  Manual  Skill  Experience  5 

Heavy  Const  Experience  4 

Installation  %  Maint 
Experience  4 


.53 

•  55 

•  6l 
.57 
.50 
.47 
.62 
.64 
•55 

•  50 


\ere  only  4  itv  ru  in  common  b<  tween  the  two  scales,  whereas  t  .0  would 
be  expected  by  chance  alone. 


Table  4 


COMPARISON  OF  ITEMS  MEETING  GIVEN  VALIDITY  CRITERIA  ACROSS 
THREE  MOS  GROUTS  AND  ITEl.E  REJECTED 


Number  of  Items 


Accepted  Rejectee.  Total 


Training  Criterion 

Chemical  Op*,  cations  Apprentice  (MOS  ‘p4A ) 
Medical  Corpsman  (MOS  91A) 

r  .06“  Above 
Below 

Wi  reman  (MGS  >*\),  Lineman  (MOS  36C), 
Engineer  Missle  Equipment  Specialist 
(MOS  o2C) 

r  . Oo*  Above 
Below 

Job  Criterion 

Pioneer  (MOS  12A),  Lineman  (MOS  f>oC ) 

r  .04*  Above 
Below 

Training  Criterion 

Chemical  Operations  Apprentice  (MOS  >'»A ), 
Medical  Corpsmon  (MOS  91A) 

r  .22b  Above 
Below 

Wireraan  (MOS  )oA ) ,  Lineman  (MOS  j>oC), 
Engineer  Mlsslc  Equipment  Specialist 
(MOS  62C) 

r  . 12b  Above 
Below 

Job  Criterion 

Pioneer  (MOS  12A),  Lineman  (MOS  juC ) , 

r  .07b  Above 
Below 


17 

3 


14 

6 


13 

T 

I 


10 

10 


11 

0 


10 

10 


8 

22 


11 

11 


12 

18 


5 

25 


4 

26 


5 

2'3 

* 


23 


23 


2b 

2b 


1? 

35 


13 

55 


13 


*  Median  validity  coefficient  of  all  Section  II  items  for  the  criterion 
indicated. 

b  70th  percentile  validity  coefficient  of  all  Section  II  items  for  the 
criterion  indicated. 


Analysis  or  particular  Job  valences  (Section  III)  revealed  little 
evidence  of  validity  for  either  training  or  Job  performance.  Table  ') 
shows  that  expressed  positive  attitude  toward  given  Jobs  lias  very  modest 
predictive  value  for  reluted  Jobs.  In  the  one  case  where  the  job  on 
which  degree  of  preference  was  expressed  is  the  same  as  the  job  on  which 
the  item  was  validated  (Liuemn),  the  validity  for  training  performance 
was  highest  (.21).  But  the  validity  for  job  performance  was  only  .05, 
and  all  other  coefficients  were  of  negligible  size  (see  Table  A -4). 

Table  ') 

VALIDITY  OF  PARTICULAR  JOB  VALENCE  FOR  MOS  PERFORMANCE 


Average  Validity  Coefficient 

Related 

MOS 

Other  MOS 

Item 

Occupation 

Related  MOS 

EBB5F& 

WSm 

51 

Carpenter,  rough 

Pioneer  (12A) 

-.n 

-.09 

-.07  .02 

54 

Heavy  Const  Spec 

Pioneer  (12A) 

•  Oo 

-.08 

-.06  -.05 

Lineman  (36c) 

.Oo 

-.01 

-.06  -.05 

58 

Electrician 

Wi reman  (36A) 

.14 

-.11 

-.02  -.14 

Lineman  (JoC) 
Engr  Msl  Sq  Sp 

.02 

.05 

-.02  -.14 

(62C) 

-.05 

-.Oo 

-.02  -.14 

59 

Lineman 

Wireman  (^>oA ) 

•  05 

-.11 

-.03  -.11 

Lineman  (3oC) 

.21 

.05 

-.03  -.11 

60 

Heat  i-  Vent  3p 

Welder  (42C) 

•09 

-.03 

-.02  .01 

Engr  Msl  Eq  Sp 
(62C) 

-.05 

.01 

-.02  .01 

Average 

.04 

-.01+ 

-.04  -.05 

Analysis  of  Job-related  pre -service  experience  (Section  IV )  also 
yielded  modest  validity  for  performance  in  MOS  which  included  ouch  duties 
(Table  6).  Again,  in  one  instance  in  which  the  experience  ms  in  the 
major  duty  of  the  MOS  (Welder),  validity  coefficients  of  .29  for  training 
and  .17  for  Job  performance  were  found.  Another  case --telephone  line- 
work  for  Linemen — yielded  a  coefficient  of  .13  for  training  and  -.01  for 
Job  performance.  The  three  experience  factors  from  the  earlier  analysis 
showed  only  a  modest  degree  of  positive  validity  for  some  MOS  training 
courses  (Table  7 ) •  In  general,  however,  results  from  analysis  of 
Sections  III  and  IV  of  AJA^-M  suggested  no  valid  scales  for  further  study. 


Table  b 


VALIDITY  OF  JOB  REIATED  EXPERIENCE  FOR  MOS  PHIFORMANCE 


Validity 

Coefficient 

Using  MOS 

Other  MOS 

Item 

Experience 

Using  MOS 

Training 

Job 

Training 

Job 

62 

Build  Concrete  forms, 
scaffolds 

Pioneer  (12A) 

.07 

-.01 

.22 

•  Ob 

66 

Construct  in  brick, 
concrete 

Pioneer  (12A) 

.01 

.07 

.07 

.02 

08 

Weld,  rivet,  bolt 

Welder  (44C  or 
42C) 

.29 

.17 

.05 

.10 

Engr  Msl  Eq  Sp 
(62C) 

.01 

-.25 

.05 

.10 

69 

Move  heavy  objects  with 
digging  gear 

Lineman  (56C) 

.02 

.11 

.09 

.02 

Engr  Msl  Eq  Sp 
(62C) 

.18 

-.15 

•  09 

.02 

72 

Cut,  bend,  thread  pipes 

Welder  (44C) 

.13 

-.05 

.14 

.04 

73 

Install,  maintain  air 
conditioners 

Engr  Msl  Eq  Sp 
(62C) 

-.08 

-.04 

.03 

-.01 

74 

Install  wiring, 
lighting 

Wi reman  (36A) 

.09 

-.08 

.04 

.Oo 

Lineman  (36c) 

.20 

.08 

.04 

.06 

75 

String  telephone 
lines,  etc. 

Engr  Msl  Eq  Sp 
(62C) 

Wireman  (36A) 

.11 

-.04 

-.10 

-.04 

.04 

.00 

.Ob 

.04 

Lineman  (36c) 

.13 

-.01 

.00 

.04 

76 

Install,  repair  heating 
ventilation 

Engr  Msl  Eq  Sp 
(62C) 

.17 

-.17 

.05 

.12 

Average 

.09 

-.03 

.07 

.05 
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SUMMARY 


Item  analysis  of  AJAq-M  against  training  and  Job  performance  in 
five  construction  and  mechanical  MQS  yielded  two  potentially  valid  scales 
from  Section  I  (specific  Job  activities)— Electrical  Training  Scale  and 
Mechanical  Job  Scales.  Additional  data  from  AJAQ-BC  on  Section  II 
which  is  the  same  in  both  instruments  yielded  a  Medical -Technical  Scale 
of  promise.  Particular  Job  valences  (Section  III)  and  Job  activity  expe¬ 
rience  reported  (Section  IV)  showed  little  predictive  value.  The  three 
empirical  scales  are  being  cross -validated  on  new  mechanical  MQ6  samples, 
along  with  the  psychological  construct  scales  obtained  from  factor 
analysis  of  item  intercorrelations  reported  elsewhere. 


APPENDIX.  Item  validity  data  on  scales  developed  for  construction  and 
mechanical  MOS 

Table  A-l 

MECHANICAL  JOB  PERFORMANCE  SCALE,  AJAQ-M:  SECTION  I 

Validity  Coefficient11 
Pioneer  Wireman  Welder 


Item 

Responses  Factor  Category* 

(12A) 

(36A1 

(42C)  Average 

19 

AB 

General 

-05 

21 

08 

08 

32 

AB 

Manual  Construction 

14 

20 

-01 

11 

34 

AB 

♦» 

02 

27 

-07 

07 

38 

AB 

It 

11 

l8 

18 

16 

41 

AB 

ft 

02 

25 

05 

11 

54 

AB 

>• 

09 

l8 

13 

13 

59 

AB 

Const.  Equip.  Operation 

02 

04 

15 

07 

6l 

AB 

ft 

05 

17 

02 

08 

64 

A 

ft 

-02 

30 

16 

15 

67 

AB 

General 

06 

11 

00 

06 

68 

A0 

11 

12 

17 

01 

10 

69 

AB 

Const.  Equip.  Operation 

-04 

06 

21 

08 

70 

AB 

»» 

01 

20 

12 

11 

71 

AB 

Manual  Construction 

04 

21 

06 

10 

79 

AB 

General 

00 

15 

02 

06 

82 

AB 

Carpentry  Tools 

-02 

13 

09 

07 

89 

AB 

t» 

-07 

10 

19 

07 

93 

AB 

ft 

11 

23 

03 

12 

98 

AB 

Const.  Equip.  Operation 

06 

22 

-05 

08 

99 

AB 

Carpentry  Tools 

02 

25 

-18 

03 

*  All  items  load  on  General  Power  Factor;  category  listed  is  secondary 
Factor.  "General"  means  no  loading  on  any  secondary  factor. 

b  Decimal  points  omitted. 
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. . ...... . 


Tuble  A -2 


ELECTRICAL  TRAINING  SCALE,  AJAU-M:  Section  I 


Item 

Response 

Factor  Category 

Wi reman 

(564) 

Lineman  Engr  Msl  Eq  Sp 

(36C)  (62C)  Average 

6 

ABC 

Danger 

01 

24 

14 

13 

16 

AB 

It 

24 

11 

20 

18 

24 

ABC 

»> 

14 

lo 

06 

12 

28 

AB 

•• 

16 

09 

08 

11 

30 

AB 

tt 

06 

14 

25 

1 5 

31 

ABC 

It 

05 

12 

23 

13 

33 

ABC 

’’ 

08 

20 

14 

14 

35 

ABC 

It 

04 

09 

20 

11 

43 

AB 

t» 

10 

22 

01 

11 

45 

ABC 

Discomfort 

04 

18 

34 

19 

52 

AB 

Danger 

21 

17 

18 

19 

76 

AB 

It 

27 

07 

11 

15 

10 

AB 

Technical 

32 

09 

-08 

11 

n 

AB 

It 

32 

17 

17 

22 

63 

AB 

•* 

34 

15 

15 

21 

88 

AB 

It 

20 

04 

14 

13 

90 

A 

•* 

GO 

18 

10 

14 

95 

AB 

It 

27 

15 

16 

19 

97 

AB 

tl 

15 

15 

15 

15 

100 

AB 

ft 

35 

lo 

lo 

22 

Decimal  points  omitted. 
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Table  A -3 

MEDICAL-TECHNICAL  SCALE,  AJAy:  SECTION  II 


Item 

Response 

Average  Validit 

v  Coefficient* 

Criterion 

Job  M0S  Criterion 

Chem-Med 

Electrical 

Construction 

Chem  Opns  Appr. 
(54A),  Med  Corps - 
man  (91A) 

Wirem&n  (30A), 
Lineman  (36c), 
Engr  Msl  Eq  Sp 
(62C) 

Pioneer  (12A), 
Lineman  (36c ) 

1 

A 

24 

lb 

06 

3 

A 

32 

15 

08 

7 

B 

21 

03 

14 

10 

A 

16 

-04 

02 

12 

A 

20 

15 

08 

15 

A 

34 

14 

20 

17 

A 

22 

01 

10 

23 

A 

l6 

13 

16 

30 

A 

26 

25 

09 

31 

B 

28 

12 

-01 

33 

A 

09 

08 

14 

35 

A 

18 

08 

04 

42 

A 

22 

01 

00 

43 

A 

07 

22 

07 

44 

B 

23 

12 

02 

45 

B 

26 

04 

-02 

47 

A 

23 

13 

13 

48 

B 

32 

07 

05 

49 

B 

19 

06 

04 

50 

B  ; 

16 

00 

00 

“  Decimal  points  omitted. 
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Table  A -A 

VALIDITY  OF  OCCUPATIONAL  VALLi'ICES,  AJA*:  SUCTION  III  FOF;  ilOS  P2RF0AI 


